measure ai
How Do You Measure AI?
Millions of people use artificial intelligence (AI) tools like ChatGPT daily to do everything from generating code to drawing images to creating business ideas. Those AI tools appear to be getting better. Back in November 2022 when it was launched, ChatGPT was powered by GPT-3.5, at the time the most powerful model offered by OpenAI. Yet GPT-3.5 was quickly eclipsed by GPT-4 just a few months later. GPT-4 crushed GPT-3.5 on a range of benchmarks, including its performance on the bar exam (GPT-4 scored in the 90th percentile; GPT-3.5 in the 10th).
Researchers propose framework to measure AI's social and environmental impact
In a newly published paper on the preprint server Arxiv.org, Through techniques like compute-efficient machine learning, federated learning, and data sovereignty, the coauthors assert scientists and practitioners have the power to cut contributions to the carbon footprint while restoring trust in historically opaque systems. Sustainability, privacy, and transparency remain underaddressed and unsolved challenges in AI. In June 2019, researchers at the University of Massachusetts at Amherst released a study estimating that the amount of power required for training and searching a given model involves the emission of roughly 626,000 pounds of carbon dioxide -- equivalent to nearly 5 times the lifetime emissions of the average U.S. car. Partnerships like those pursued by DeepMind and the U.K.'s National Health Service conceal the true nature of AI systems being developed and piloted.
- North America > United States > Massachusetts (0.25)
- North America > Canada > Quebec > Montreal (0.16)
- Health & Medicine > Health Care Providers & Services (0.56)
- Law > Environmental Law (0.41)
Forget the Turing Test: Here's How We Could Actually Measure AI
A chatbot pretending to be a 13-year-old Ukrainian boy made waves last weekend when its programmers announced that it had passed the Turing test. But the judges of this test were apparently easily fooled, because any cursory exchange with'Eugene Goosterman' reveals the machine inside the ghost. Maybe the time has come, 60 years after Alan Turing's death, to discard the idea that imitating human conversation is a good test of artificial intelligence. "I start my Cognitive Science class with a slide titled'Artificial Stupidity,'" said Noah Goodman, director of the computation and cognition lab at Stanford University. "People have made progress on the Turing test by making chatbots quirkier and stupider."
- Education (0.53)
- Leisure & Entertainment > Games (0.31)
Why we need to measure AI's societal impacts
Artificial Intelligence produces compelling advances in complex tasks and dramatic improvements in areas such as energy consumption. At the same time, it is already making problematic judgements that have significant impacts on our daily lives, prompting the need for a research field that measures and assesses the effects of current AI systems.